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Abstract 

In testimony before U.S. Congress on March 11, 2020, members of the House Oversight and 
Reform Committee were informed that estimated mortality for the novel coronavirus was ten- 
times higher than for seasonal influenza. Additional evidence, however, suggests the validity of 
this estimation could benefit from vetting for biases and miscalculations. The main objective of 
this article is to critically appraise the coronavirus mortality estimation presented to Congress. 
Informational texts from the World Health Organization and the Centers for Disease Control 
and Prevention are compared with coronavirus mortality calculations in Congressional 
testimony. Results of this critical appraisal reveal information bias and selection bias in 
coronavirus mortality overestimation, most likely caused by misclassifying an influenza 
infection fatality rate as a case fatality rate. Public health lessons learned for future infectious 
disease pandemics inelude: safeguarding against research biases that may underestimate or 
overestimate an associated risk of disease and mortality; reassessing the ethics of fear-based 
public health campaigns; and providing full public disclosure of adverse effeets from severe 
mitigation measures to contain viral transmission. 


Keywords: coronavirus mortality overestimation; COVID-19; sampling bias; case fatality rate; 
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Introduction 


On September 23, 1998, the U.S. National Aeronautics and Space Administration (NASA) 
permanently lost contad with the $125 million Mars Climate Orbiter.' A simple miscalculation, 
failure to convert English measurements to metric measurements, doomed the Mars space 
mission. A later investigation found that backup quality assurance procedures were not in place 
at NASA to catch and correct this simple miscalculation. Fast forward 22 years to another crisis 
involving a U.S. government agency: On March 11, 2020, the U.S. Congress House Oversight 
and Reform Committee received Information from the National Instituto of Allergy and 
Infectious Diseases (NIAID) conceming the novel coronavirus, severo acute respiratory 
syndrome coronavirus 2 (SARS-CoV-2), and coronavirus-disease 2019 (COVID-19). Based on 
the data available at the time, Congress was informed that the estimated mortality rate for the 
coronavirus was ten-times higher than for seasonal influenza, which helped launch a campaign of 
social distancing, organizational and business lockdowns, and shelter-in-place orders. 

Previous to the Congressional hearing, a less severo estimation of coronavirus mortality 
appeared in a February 28,2020 editorial released by NIAID and the Centers for Disease Control 
and Prevention (CDC). Published online in the New England Journal of Medicine (NEJM.org), 
the editorial stated: 

“...the overall clinical consequences of Covid-19 may ultimately be more akin to those of a 
severo seasonal influenza (which has a case fatality rate of approximately 0.1%).”"^ 

Almost as a parenthetical afterthought, the NEJM editorial inaccurately stated that 0.1% is the 
approximate case fatality rate of seasonal influenza. By contras!, the World Health Organization 
(WHO) reported that 0.1% or lower is the approximate influenza infection fatality rate,^ not the 
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case fatality rate. To fully appreciate the significance of discrepancies in fatality rate usage by 
NIAID, the CDC, and the WHO, brief definitions of relevant epidemiological terms follow. 

Case fatality rates (CFRs), infection fatality rates (IFRs), and mortality rates are used by 
epidemiologists to describe deaths during and after an infectious disease outbreak. The CDC 
defined a mortality rate as the frequency of deaths within a time period relative to the size of a 
well-defined population ® Patients may be classified as having an influenza-like illness (ILI) such 

n 

as COVID-19 according to standard criteria in a case definition. A case fatality rate (CFR) is 
defined as the proportion of deaths among confirmed cases of the disease. CFRs indicate the 
disease severity, while an infection fatality rate (IFR) is defined as the proportion of deaths 

o 

relative to the prevalence of infections within a population. IFRs are estimated following an 
outbreak, often based on representative samples of blood tests of the immune system in 
individuáis exposed to a virus. Estimation of the IFR in COVID-19 is urgently needed to assess 
the scale of the coronavirus pandemic.^ 

Because different types of fatality rates can vary widely, it is imperative to not confuse 
fatality rates with one another; else misleading calculations with significant consequences could 
result. Interestingly, a search of the keyword term “infection fatality rate” on the CDC website 
retumed no matching results, ñor was the epidemiological term located in the 511-page CDC 
publication, Principies of Epidemiology in Public Health Practice. This terminology omission, in 
conjunction with questionable use of fatality rate terminology in the NEJM editorial, raises red 
flags—warning of possible inaccuracies in the coronavirus mortality estimation presented to 
Congress. Similar to the need to vet for miscalculations that might have rescued NASA’s 1998 
Mars mission, vetting the coronavirus mortality estimation for miscalculations and biases may 
benefit the validity of mortality conclusions. Therefore, the purpose of this article is to present an 
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ad hoc critical appraisal of the coronavirus mortality estimation presented to U.S. Congress on 


March 11,2020. 


Main 

Findings from a comparative analysis of selected video and texts are used in this article to 
critically appraise the validity of coronavirus mortality calculations presented in U.S. 
Congressional testimony. Critical appraisal is a process that judges the validity of scientific 
research evidence.'*’ Comparative analysis is a tool used in a grounded theory methodology to 
investígate an unexplored area through logical induction of coherent themes and explanations 
that are grounded in empirical evidence." Text from the February 2020 NEJM.org editorial and 
video of Congressional testimony are compared with reliable informational texts from the WHO 
and CDC. Inconsistencies, inaccuracies, Fiases, utilization, and consequences of the coronavirus 
mortality estimation are discussed. 

In NIAID testimony before the House Oversight and Reform Committee Hearing on 
Coronavirus responso, Day 1, the Committee learned that mortality from seasonal influenza is 
0.1%. Additionally, it was reported to Congress that the overall coronavirus mortality of about 
2-3% had been reduced to 1% to take into account infected people who are asymptomatic or 
have mild symptoms. The adjusted mortality rate from coronavirus of 1% was then compared 
with the 0.1% mortality rate from seasonal influenza, and the conclusión was reported to the 
House Committee that the coronavirus was ten-times more lethal than seasonal influenza. 

In a comparative analysis with WHO and CDC documents, the coronavirus mortality rate of 
2-3% that was adjusted to 1% in Congressional testimony is consistent with the coronavirus 
CFR of 1.8-3.4% (median 2.6%) reported by the CDC.'^ Furthermore, the World Health 
Organization reported that the CFR of the HlNl influenza virus is also 2-3%, similar to the 
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unadjusted 2-3% CFR of the coronavirus reported in Congressional testimony, with no 
meaningful difference in mortality. As previously mentioned, the World Health Organization 
also reported that 0.1% is the IFR of seasonal influenza,^ not the CFR of seasonal influenza as 
reported in the NEJM editorial. 

Discussion 

Confusión between case fatality rates and infection fatality rates may seem trivial, and it is 
easy to overlook at first, but this confusión may have ultimately led to an unintentional 
miscalculation in coronavirus mortality estimation. IFRs from samples across the population 
inelude undiagnosed, asymptomatic, and mild infections, and are often lower compared to CFRs, 
which are based exclusively on relatively smaller groups of moderately to severely ill diagnosed 
cases at the beginning of an outbreak. Due to host defense mechanisms and autoimmunity 
provided by innate and adaptive immune responsos,'"^ asymptomatic infections are often 
prevalent in influenza.'^ With many asymptomatic infections already identified in COVID-19,'^ 
it appears unlikely that the IFR in an influenza-like illness like COVID-19 would approximate 
the disease’s CFR. Presymptomatic infections can also lower the proportion of asymptomatic 
infections. For example, a CDC report found that asymptomatic individuáis identified through 
reverse transcriptase-polymerase chain reaction (PCR) testing developed symptoms a week later, 

17 

and those individuáis were re-classified as having been presymptomatic at the time of testing. 

In Figure 1, four cases grouped in the dotted-line box are also included among seven 
infections, illustrating that all cases are infections but not all infections are cases, a potential 
point of confusión in media reports of COVID-19. For example, a high number of coronavirus 
infections were discovered in U.S. meat-packing plants in lowa, but these infections were 
reported as cases in the media,'^ potentially causing a type of information bias known as 
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misclassification?*’ Misclassification refers to “the erroneous classification of an individual, a 
valué, or an attribute into a category other than that to which it should be assigned." This type 
of Information bias in epidemiological research can lead to underestimates or overestimates of 
associated disease and mortality risks?*' 

Case Fatal íty Rate 

1 Fatal íty 

iiilii 

4 Cases 

Infection Fatalíty Rate 

1 Fatali^ 

7 infectíons 

Figure 1, Case fatality rate and infection fatality rate, 

1 fatality / 4 cases = 25% case fatality rate. 

1 fatality / 7 infectíons = 14.28% infection fatality rate. 

CFRs and IFRs represent different segments of a targeted population and contain widely 

different proportions of nonfatal infections; therefore, misapplying findings or generalizing 

22 

inferences between these two groups can cause a type of selection bias known as sampling bias 
or ascertainment bias. In this type of bias, people do not represent segments of the population 
to whom findings apply. Furthermore, “...comparisons of the CFR of one disease with the IFR 
of another are mostly useless,” and sampling bias can lead to serious inaccuracies, as when 
Congress was informed that the coronavirus is ten-times more lethal than seasonal influenza. 
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A comparison of coronavirus and seasonal influenza CFRs may have been intended during 
Congressional testimony, but due to misclassifying an infection fatality rate as a case fatality 
rate, the comparison turned out to be between an adjusted coronavirus CFR of 1% and an 
influenza IFR of 0.1%. Had the adjusted coronavirus mortality rate not been lowered from 3% to 
1%, fatality comparisons of the coronavirus to the IFR of seasonal influenza would have 
increased from 10-times higher to 20- to 30-times higher. By then, epidemiologists might have 
been alerted to the possibility of a miscalculation in such an alarming estimation. 

Quality assurance 

Most people rely on trusted public health experts from organizations like the CDC to 
disseminate vital Information on infectious diseases. Unfortunately, even experts can make 
simple miscalculations which can lead to catastrophic results. In the example of NASA’s lost 
Mars Climate Orbiter, the NASA board investigating the failed mission recognized that mistakes 
happen on projects, "However, sufficient processes are usually in place on projects to catch these 
mistakes before they become critical to mission success." The NASA board also recognized the 
importance of quality assurance procedures to prevent future failures. Of relevance, in 2018, the 
National Institute of Neurological Disorders and Stroke (NINDS) provided an exemplary 
definition of quality assurance (QA) in clinical and health Sciences: 

"The objectives of QA procedures are to assure the accuracy and consistency of study data, 
from the original observations through the reporting of results and to ensure that study 
results are considered valid and credible within the scientific and clinical communities."^^ 

Similar to NASA’s quality assurance problems in 1998, quality assurance procedures at U.S. 
national public health organizations in 2020 may benefit from review and revisión to prevent 
crucial mortality miscalculations of infectious diseases in the future. As a safeguard against 
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misuse of fatality rates, and protection in the event of non-standardized or inter-organizational 
discrepancies in terminology, every fatality rate should clearly define the denominator of the rate 
as the specific group to whom fatalities apply, either to the total population in mortality rates, 
confirmed cases of a disease in CFRs, or individuáis exposed to a viral infection in IFRs. 

Mitigation measures 

As the campaign to mitigate coronavirus transmission was implemented from March into 
May, 2020, expected coronavirus mortality totals in the United States appeared much lower than 
the overestimation reported in Congressional testimony on March 11. Compared to the most 
recent season of severe influenza A (H3N2) in 2017-2018,^^ with 80,000 U.S. deaths reported by 
CDC officials,^^ U.S. coronavirus mortality totals had just reached 80,000 on May 9, 2020.^^ By 
then, relativo to the 2017-2018 influenza, it was clear that the coronavirus mortality total for the 
season would be nowhere near 800,000 deaths inferred from the ten-fold mortality 
overestimation reported to Congress. Even after adjusting for the effect of successful mitigation 
measures that may have slowed down the rate of coronavirus transmission, it seems unlikely that 
so many deaths were completely eliminated by a non-pharmaceutical intervention like social 
distancing, which was only intended to contain infection transmission, not suppress infections 
and related fatalities.^^ Also in early May, 2020, a New York State survey of 1,269 COVID-19 
patients recently admitted to 113 hospitals found that most of the patients had been following 
shelter-in-place orders for six weeks, which raised State officials’ suspicions about social 
distancing effectiveness. Still, polis showed the public credited social distancing and other 
mitigation measures for reducing predicted COVID-19 deaths, and for keeping people safe from 

32 33 

the coronavirus. ’ 
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Surprisingly, disproportionate mortality increases in Italian and American healthcare 
facilities during the height of the COVID-19 outbreak were not unique; similar healthcare 
facility crises occurred during the 2016-2017 influenza season in Italy,^"^ and during the 2017- 
2018 influenza season in the United States. Yet, these earlier outbreaks did not appear to 
receive the same intensive media coverage as COVID-19. Although media reports of new 
coronavirus infections reinforced the public’s belief that the virus was continuing to spread, 
greater levels of testing may have increased detection of infections that were already prevalent 
throughout the population. In addition, the accuracy of coronavirus tests rushed into production 
during the pandemic were unknown.^® PCR testing has been in use since the detection of the A 
(H5N1) influenza virus in 2005, but a serious limitation of PCR testing is that nucleic acid 
detection is not capable of determining the difference between infective and non-infective 
viruses. Moreover, the CDC modified criteria to record coronavirus mortality by including 
“probable” and “likely” deaths in the International Classification of Diseases code (ICD) for 
COVID-19.^‘^ 

By June 21, new daily deaths from the coronavirus dropped to 267 in the United States, a 
90% decrease from 2,693 daily deaths reported on April 21. However, confirmed cases in some 
areas increased as lockdowns lifted,"^*’ and total U.S. infections had reached 1,254,055 by June 
21. Several reasons in addition to increased viral transmission could account for case increases. 
For example, ill people may no longer fear going to hospitals as society reopens,"^' and 
coronavirus testing may also result in greater differential diagnosis of SARS-CoV-2 infections 
from other common respiratory viral infections. With more reported cases of COVID-19 in 
younger people following reopening,"^^ case fatality rates could actually decline due to lower 
associated mortality risk in this age group. Furthermore, country comparisons of coronavirus 
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case fatality rates are often confounded by numerous factors,"^"^ including: healthcare differences 
in case definitions, access to quality treatment and reliable testing, compliance with mitigation 
measures, and underlying health condi tions; demographic differences in age, race, 
socioeconomic status, and population density; and geo-political differences including climate, 
seasonality, environmental pollution, social inequities and unrest, personal liberties, public health 
policies, reliability in reporting valid government statistics of disease, and lifestyle customs that 
affect physical and mental health, public sanitation, and personal hygiene. Ultimately, with a 
myriad of uncontrolled confounding factors, a serosurvey of representativo samples of a 
population is a more reliable method to determine the true prevalence of coronavirus infections. 

Emerging confounding factors in the United States have also contributed to a rising 
mortality trend in ILIs such as COVID-19. For example, each year surviving members of the 
ageing Baby-Boomer cohort of 76 million people born between 1946 and 1964 enter the high- 
risk category for ILIs, increasing the burden placed on healthcare SystemsAlso, research 
shows that a warming trend in the Artic can lead to more extreme winter weather condi tions, 
especially in the Eastern United States,"^^ which may play a role in rising mortality rates from 
ILIs during the influenza season. 

As health authorities responded to the COVID-19 pandemic by implementing lockdowns 
and other mitigation measures with minimal supporting evidence, scientists wamed of “a fiasco 
in the making,”"^^ Caution was also raised against violations of fundamental principies of Science 
and logic, such as the mistaken assumption that correlation implies causation."^"^ For example, the 
public’s belief that mitigation measures were responsible for reducing coronavirus mortality may 
be a post hoc fallacy if lower mortality was actually due to the overestimation of coronavirus 
deaths. Furthermore, implementing the unconfirmed hypothesis that mitigation measures save 
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lives in vulnerable populations, and rejecting the nuil hypothesis that assumes no life-saving 
effect exists, is a type I error in hypothesis testing. The nuil hypothesis does not assume a priori 
knowledge. Therefore, before implementing mitigation measures that incur severe costs, the onus 
is on mitigation proponents to formally reject the nuil hypothesis by justifying claims of life- 
saving benefits. Additionally, education in principies of basic research methods is essential for 
consumers of public health research, and there is a need to increase instruction in the Science and 
logic of research methods in general education curricula."^^ More research of non-dmg mitigation 
interventions is also urgently needed to prevent COVID-19, especially in vulnerable 
populations.^*’ 

Scientists also warned of public health decisions made without reliable data of infection 
prevalence within the population."*^’"*^ Lacking valid input data due to insufficient testing for 
disease prevalence, statistical modelling methods often relied on speculative assumptions, 
producing fearful predictions of increased mortal! ty which have often pro ved unreliable.^’ A 
systematic review found that most diagnostic and predictive models for COVID-19 lack rigor, 
have a high risk of selection bias, and are likely to have lower predictive performance in actual 
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practice compared to optimistic reports published in the research literature. 

A revised versión of a non-peer-reviewed study on COVID-19 antibody seroprevalence in 
Santa Clara County, California found that infections were many times more prevalent than 
confirmed cases. As more serosurveys are conducted throughout the country, a nationally 
coordinated COVID-19 serosurvey of a representativo sample of the population is urgently 
needed,^"* which can determine if the national infection fatality rate is low enough to expedito an 
across-the-board end to restrictivo mitigating measures. Plans for a national U.S. serosurvey 
were announced in April 2020 by the National Institutos of Health, to be conducted by NT A ID 
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and the National Instituto of Biomedical Imaging and Bioengineering (NIBIB), with the 
assistance of the National Center for Advancing Translational Sciences (NCATS) and the 
National Cáncer Instituto (NCI).^^ Of relevance, nationwide mitigation measures such as 
lockdowns, social distancing, and shelter-in-place orders were not implemented during the 2017- 
2018 influenza with 45 million U.S. illnesses reported by the CDC.^^ Neither were mitigation 
measures implemented during the 2009 influenza, with reported estimates adjusted for 
underreported hospitalizations of approximately 60.8 million U.S. cases, ranging between 43.3 
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million to 89.3 million cases. 

Fear and collateral damage 

Psychological adverse effects such as anxiety, anger, and post-traumatic stress have been 
linked to restrictivo public health mitigation measures due to isolation, frustration, financial loss, 
and fear of infection.^^’^^ A June 8,2020 survey from the Association for Canadian Studies found 
that fear of contracting the coronavirus affected 51% of the Canadian population, compared to 
56% of the U.S. population.^® Venturing out into public during the reopening phase of the 
lockdown was stressful to 50% of Canadians compared to 56% of Americans. A second wave of 
the virus was also expected by 76% of Canadians and 64% of Americans. Furthermore, the 
possibility exists that yet another novel virus could emerge, potentially reigniting a perpetual 
process of unfounded fear and unnecessary lockdowns if mortality estimations are not properly 
vetted. 

Fear, in contrast to moral civic duty and political orientation, was shown to be a more 
powerful predictor of compliance with mitigating behavior in responso to a viral pandemic, but 
with decreasing wellbeing and poorer decisión making.®' Studies have shown that fear impairs 
performance of cognitive tasks through debilitating anxiety and worry.®^ Even if a threat ceases 
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to exist, prolonged fearful avoidance of threats is maladaptive and restricts a retum to normal 
social interaction and productivity.®^ For example, after the outbreak of severe acute respiratory 
syndrome (SARS) had ended in 2004, avoidance behavior continued to restrict people’s social 
interactions and prevented people from returning to woik.^*^ 

Exaggerated levels of fear were driven by sensationalist media coverage during the COVID- 
19 pandemic."^"^’®^’®® And yet, while the public was ordered to lockdown, overall costs and 
benefits to society from severe mitigation measures had not been assessed."^"^ Fear of infection 
also prevented people from seeking needed healthcare Services in hospitals during the 
pandemic.^^ The ethics of implementing fear-based public health campaigns needs to be 
reevaluated for the potential harm these strategies can cause Dissemination of vital health 
Information to the public should employ emotionally persuasive messaging without exploiting 
and encouraging overreactions based on fear. 

In addition, legal and ethical violations associated with mitigation of pandemic diseases 
were previously investigated by the Institute of Medicine in 2007.^^ People should have the right 
to full disclosure of all Information pertinent to adverse impacts of mitigation measures during a 
pandemic, including information on legal and constitutional human rights issues,"^"^ and the public 
should be guaranteed a voice in a transparent process as authorities establish public health 
policy. 

Lastly, severe mitigating measures during the COVID-19 pandemic caused considerable 
global social and economic disruption. Enforced lockdowns increased domestic violence, 
closed businesses and schools, laid off workers, restricted travel, affected capital markets, 
threatened the security of low-income families, and saddled governments with massive debt. 
Between Eebruary and April 2020, U.S. unemployment rose from 3.5%, the lowest in 50 years. 
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to 14.7% 7' A recession in the United States was also officially declared in June 2020 by the 
National Bureau of Economic Research, ending 128 months of historie economic expansión. Of 
relevance, economic downturns are associated with higher suicide rates compared to times of 
prosperity, and increased suicide risk may be associated with economic stress as a consequence 
of severe mitigation measures during a pandemic. Relapsos and newly diagnosed cases of 
alcohol use disorder were also predicted to increase due to social isolation, and harmful drinking 
in China increased two-fold following the COVID-19 outbreak. As a global natural experiment, 
psychological outeomes from restrictive interventions in the COVID-19 pandemic require further 
investigations.^"^ 


Information Bias 

Misclassifying viral infections as cases 


Sampling Bias 

MisappIyingIFRs to cases, or CFRs to infections 


Virus Mortality Overestimation 

Miscalculatingthat a virus is mucFi more lethal than seasonal influenza 


Fear and Containment 

Severe mitigation toslow, notsuppress, viral infection 


Psychosocial harm and economic disruption 

Adverse effeets of lockdowns witFi questionable benefitsand high costs 


Figure 2, Blases and potential related effeets of virus mortality overestimation. 
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Public health lessons leamed during the COVID-19 pandemic con tribute knowledge and 
insights that can be applied to prevent future public health crises. Figure 2 shows a flow chart 
that summarizes biases and potential effects of viral mortality overestimation observed in a 
pandemic. Failure to intervene at the source of the problem, at the upstream levels of information 
bias and sampling bias, can allow fear to rapidly escálate and may cause an overactive response 
that produces severely harmful collateral damage to society. 

Conclusión 

Sampling bias in coronavirus mortality calculations led to a ten-fold increased mortality 
overestimation in March 11, 2020 U.S. Congressional testimony. This bias most likely followed 
from information bias due to misclassifying a seasonal influenza infection fatality rate as a case 
fatality rate, evident in a NEJM.org editorial. Evidence from the World Health Organization 
confirmed that the approximate case fatality rate of the coronavirus is generally no higher than 
that of seasonal influenza. By early May, 2020, mortality levels from COVID-19 were 
considerably below predicted overestimations, a result which the public attributed to successful 
mitigating measures to contain the spread of the novel coronavirus. 

This article presented importan! public health lessons learned from the COVID-19 
pandemic. Reliable safeguards are needed in epidemiological research to prevent seemingly 
minor miscalculations from developing into disasters. Sufficient organizational quality assurance 
procedures should be implemented in public health institutions to check, catch, and correct 
research biases and mistakes that underestimate or overestimate associated risks of disease and 
mortality. Particularly, the denominator of fatality rates should clearly define the group to whom 
fatalities apply. Public health campaigns based on fear can have harmful effects, and the ethics of 
such campaigns should be reevaluated. People need to have a greater voice in a transparent 
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process that influences public health policy during an outbreak, and educational curricula should 
inelude basic research methods to teach people how to be better consumers of public health 
Information. The public should also be fully informed of the adverse impaets on psychological 
well-being, human rights issues, social disruption, and economic costs associated with restrictive 
public health interventions during a pandemic. 

In closing, nations across the globe may fearfully anticipate future waves of the coronavirus 
pandemic, and look bleakly toward outbreaks of other novel viral infections with a return to 
severe mitigation measures. However, well-worn advice from a famous aphorism by the poet 
philosopher George Santayana should be borne in mind, which is relevant to public health 
lessons learned in this article: “Those who cannot remember the past are condemned to repeat 
it.’’^' 
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